PR Checklist

• Use descriptive commit messages.
• Provide tests for your changes.
• Update any related documentation and include any relevant screenshots.
• Check if changes need to be made to docs (README or any guides in /docs/).

What type of PR is this? (check all applicable)

• Refactor
• Feature
• Bug Fix
• Optimization
• Documentation Update

Description

The changes are part of an "OLMES matching" effort. The changes include:

• csqa.py: CommonsenseQAMC_OLMES: Sets FEWSHOT_SPLIT = "train" (was inheriting "validation" from parent)
• medqa.py: MedQAMC_OLMES: Sets FEWSHOT_SPLIT = "train" (was inheriting "dev" from parent)
• sciq.py: SCIQ_OLMES: Sets FEWSHOT_SPLIT = "train" (was inheriting "test" from parent)
• piqa.py: PIQA_OLMES: Sets FEWSHOT_SPLIT = "train" and changes instruction text from "Question: {goal}\n..." to "Goal: {goal}\n...", which is what OLMES is doing.
• drop.py:
  • New class DropCompletion_OLMES: Subclass of DropCompletion with FEWSHOT_SPLIT = "train" and max_tokens = 100 (vs. 50 in base)
  • DropMC: Added _get_cue_text returning "Answer:" (previously inherited empty string "" from BaseTask)
• If OLMES evaluates on all splits, we select the largest of those for our _OLMES variant. This affects csqa, piqa, sciq, and social_iqa.
• social_iqa.py: Added missing "Answer:" prefix to fewshot samples, via _get_fewshot_target_text
• task_names.py: Registers DropCompletion_OLMES
• task-prompts-hashes.json: Added hash for DropCompletion_OLMES and updated hash of tasks whos fewshot split changed.
• Update documentation.

Added/updated tests?

• Yes
• No, and this is why: please replace this line with details on why tests
  have not been included
• I need help with writing tests